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METHOD AND APPARATUS FOR BRIEF DESCRIPTION OF THE DRAWINGS 

MODIFYING SCREEN RESOLUTION BASED n 

ON AVAILABLE MEMORY present invention is illustrated by way of example 

and not limitation in the figures of the accompanying 

BACKGROUND OF THE INVENTION ^ drawings, in which like references indicate similar elements 

1. Field of the Invention and in which: 

The present invention pertains to video processing. More FIG. 1 illustrates a display of a video multiplexing system 

particularly, this invention relates to modifying screen reso- according to one embodiment of the present invention; 

lutions. piQ 2 illustrates a video multiplexing system according 

2, Background 10 to one embodiment of the present invention; 

The display of video images has become extremely FIGS. 3, 4, and 5 are a more detailed illustration of the a 

important in the modern world, including typical computer video multiplexing system of FIG. 2 according to one 

uses, such as multimedia games and video conferencing, as embodiment of the present invention; 

well as other entertainment uses, such as movies. Modern ™„ * •« * . £ , . 

„ -11 -i ui a • FIG, 6 illustrates a video format analyzer according to one 

computers, as well as other commercially available devices, 15 . . r . . J & 

* • « • 1 j j. 1 . embodiment of the present invention; 

typically include the ability to display video images, many r 

of which allow display of real-time motion video. Examples FIG. 7 is a flowchart illustrating the steps associated with 

of such systems include display devices designed for use establishing a video locked state between two signals 

with typical computer systems. according to one embodiment of the present invention; 

One problem encountered in many systems is the amount 20 Fia 8 illustrates a storage medium including one 
of memory required to display video images. Mass storage embodiment of the present invention; 
devices, such as CD-ROMs, digital video disks (DVDs) and FIG. 9 is a flowchart illustrating the steps followed in 
hard disk drives are relatively inexpensive. However, the modifying a horizontal screen resolution according to one 
process of displaying video images typically requires trans- embodiment of the present invention; 
fer of these images from the mass storage device (or other 25 FIG. 10 illustrates multiple scan lines of a screen accord- 
source) to a more expensive random access memory (RAM), mg t0 one embodiment of the present invention; 
typically referred to as a frame ^^t^MA^^ FIG. 11 is a timing diagram illustrating pixel clocks 
frame buffer is often video RAM (VRAM), which is par- accordin l0 one embodiment of the present invention; 
ticularly suited to displaying video, but which is also very _ ¥ _ . P „ 
expensive. The amount of VRAM within a system can often 30 . A FIG ; 12 15 a Rowc }™} illustrating the steps followed in 
noticeably affect the overall system price. Thus, it would be identifying a proper horizontal resolution according to one 
beneficial to provide a way to reduce the amount of VRAM embodiment of the present invention; and 
necessary for video images. FIG. 13 illustrates a display according to an alternate 

Additionally, modern computer systems typically provide embodiment of the present invention, 

users with the ability to alter screen resolution and color 35 DETAILED DESCRIPTION 
resolution for their display devices. The screen resolution 

refers to the number of pixels displayed by the display ^ &c following detailed description numerous specific 
device, while the color resolution refers to the number of details are set forth in order to provide a thorough under- 
different colors which each pixel can have. Larger screen standing of the present invention. However, it will be 
and color resolutions provide for more realistic video 40 understood by those skilled in the art that the present 
images, but also require additional memory, including invention may be practiced without these specific details. In 
VRAM, for storage and display. Thus, it would be beneficial other instances well known methods, procedures, 
to provide a way to reduce the amount of memory used for components, and circuits have not been described in detail 
video images without significantly reducing the image qual- so as not to obscure the present invention, 
ity. 45 Some portions of the detailed descriptions which follow 
Thus, a need exists for a method and apparatus for are presented in terms of algorithms and symbolic rep re - 
modifying screen resolution based on available memory. sentations of operations on data bits within a computer 

„ , :r n „ „ „„ r __ T _ memory. These algorithmic descriptions and representations 

SUMMARY OF THE INVENTION are the y means by lhose skil £ d in [he ^ processing 

A method and apparatus for modifying screen resolution 50 arts to most effectively convey the substance of their work 

based on available memory is described herein. According to others skilled in the art. An algorithm is here, and 

to the present invention, a first screen resolution for a display generally, conceived to be a self-consistent sequence of steps 

is first identified. Then, a second screen resolution for at leading to a desired result. The steps are those requiring 

least a portion of the display is created, wherein the creating physical manipulations of physical quantities. Usually, 

is based on the first screen resolution and an amount of 55 though not necessarily, these quantities take the form of 

memory to be used for the at least a portion of the display. electrical or magnetic signals capable of being stored, 

According to one embodiment, the first screen resolution transferred, combined, compared, and otherwise manipu- 

corresponds to a video signal received from a computer lated. It has proven convenient at times, principally for 

system while the second screen resolution corresponds to a reasons of common usage, to refer to these signals as bits, 

motion video signal received from a remote video confer- 60 values, elements, symbols, characters,* terms, numbers, or 

encing camera. In this embodiment, the second screen thelike.lt should be borne in mind, however, that all of these 

resolution is determined based on the amount of frame buffer and similar terms are to be associated with the appropriate 

memory available for the display of the motion video signal. physical quantities and are merely convenient labels applied 

Also, the motion video signal is overlaid on the video signal to these quantities. Unless specifically stated otherwise as 

received from the computer system, allowing both images to 65 apparent from the following discussions, it is appreciated 

be displayed concurrently despite the differing screen reso- that throughout the present invention, discussions utilizing 

lutions. terms such as "processing" or "computing" or "calculating" 
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or "determining" or "displaying" or the like, refer to the purpose of compression, encoding, or other image process- 
action and processes of a computer system, or similar ing purposes. For example, "video content" could be sepa- 
electronic computing device, that manipulates and trans- rated from its "video envelope" and multiplexed with 
forms data represented as physical (electronic) quantities another "video signal" for a picture-in -picture (PIP) effect, 
within the computer system's registers and memories into 5 "Video envelope" is used herein to mean a composition of 
other data similarly represented as physical quantities within signal components with specific timing relationships used to 
the computer system memories or registers or other such convey a video image between two points. Signal compo- 
information storage, transmission or display devices. nents typically include horizontal sync and vertical sync. 
Definitions "Video content" is inserted into the "video envelope" to 

"Color space" is used herein to mean a representation of 10 result in a complete "video signal". If no "video content" is 

a set of colors. Typical color spaces include RGB, Y1Q, inserted into the "video envelope", the resulting "video 

YUV and YCrCb. signal" appears as a solid black image when viewed on a 

"Computer video" and "computer video signal" are used display device, 

herein to mean an analog RGB video signal or signals "Video signal" is used herein to mean a composite signal 

generated by any computer or similar hardware device. 15 containing "video content" within a "video envelope". 

Further, such term as used herein includes both "video Overview of the Video Multiplexing System 

content" as well as format and timing information that is FIG. 1 illustrates a display of a video multiplexing sys- 

separate from the "video content". tern. A display 100 is illustrated which includes windows 

"Far side image" is used herein to mean the motion video 102, 104, and 106. Such windows may be generated by a 

image sent from a remote site or sites participating in a video 20 video conferencing application, such as the local and remote 

conferencing session. windows, or by any of a wide variety of other applications, 

"Group clock" or "GCLK" is used herein to mean a such as a game or other application. It should be noted that 

derivative of the "synthesized pixel clock". According to virtually any number of windows can be displayed on the 

one embodiment, the GCLK is the "pixel clock" divided by display 100, and that a window could be enlarged to be full 

four. 25 screen (that is, encompass the entire display). 

"Horizontal total" or "H tora/ " is used herein to mean the The display 100 has an associated screen and color 

number of pixel groups contained in a single horizontal scan resolution. The screen resolution refers to the number of 

line. U totaI is the sum, as measured in pixel groups, of the pixels which are displayed on the screen and the color 

horizontal pulse sync width, the horizontal back porch resolution refers to the number of different colors which 

("H^ ") the horizontal active time, and the horizontal 30 each pixel can be. Conventional display devices typically 

front porch ("H^, p0 rch\ support a wide range of screen and color resolutions. 

"Motion video signal" is used in the illustrated embodi- Although a large number of screen resolutions exist, typical 

ment to mean the "video signal" originating in the video screen resolutions include 640x480 (307,200 pixels), 800x 

conferencing system outside of the computer via a video 600 (480,000 pixels), 1024x768 (786,432 pixels), 1280x 

camera, VCR, television, etc, However, as discussed in more 35 1024 (1310,720 pixels), and 1600x1200 (1,920,000 pixels), 

detail below, the zooming of the present invention can use Similarly, although a large number of color resolutions exist, 

any video signal. typical color resolutions include 8 -bit (256 colors), 16-bit 

"Multiplexing" is used herein to mean the act of combin- (65,536 colors), 24-bit (16,777,216 colors), and 32-bit 

ing separate sub-images each of which may have differing (4,294,967,296 colors). 

"video envelopes". 40 The amount of storage space required for an image can be 

"Near side image" is used herein to mean the motion calculated by multiplying the number of pixels by the color 

video image generated by a local camera in a video confer- resolution. By way of example, 1024x768 screen resolution 

encing environment. It is optionally displayed on the local and 16-bit color resolution requires 1,572,864 bytes, which 

screen. equals 1.5 Megabytes (MB). 

"Pixel clock" is used herein to mean the periodic digital 45 FIG. 2 illustrates a video multiplexing system according 

clock associated with the "computer video signal" or other to one embodiment of the present invention. A computer 210 

"video signal" from which all components of the "video as illustrated can be any of a wide variety of general purpose 

signal" are referenced. It is preferably a high frequency computers, such as a desktop PC, laptop PC, or a network 

signal, the rate of which defines when the pixels are updated PC-like device. Computer 210 provides an RGB analog 

as well as how many pixels are on a horizontal scan line. A 50 computer video signal 212 (including image, timing and 

derivative of the "pixel clock" known as "group clock" is format information) to the multiplexing system 213 of the 

used to create the horizontal sync component of the "video present invention. An additional data and control path 215 is 

envelope" of the computer video signal. "Original pixel established, such as by coupling the computer's parallel port 

clock" is used herein to mean the pixel clock of the source to the video multiplexing system 213. This data and control 

of the "video signal", while "synthesized pixel clock" is 55 path 215 is used to send messages to and from the computer 

used herein to mean that pixel clock generated by a phase- 210 and the video multiplexing system 213. Alternatively, 

locked loop which is iteratively adjusted so that it tracks, but other uni-directional and/or bi-directional ports, such as a 

is not necessarily equal to, the "original pixel clock". serial port, Universal Serial Bus (USB) port, other IEEE 

"Pixel group" is used herein to mean four pixels, although 1394-compliant ports, etc., may be used for the data and 

it will be understood that in other embodiments, a different 60 control path 215. 

number may instead be used. The video multiplexing system 213 combines the com- 
"Video content" or "video information" is used herein to puter video signal 212 generated from the computer 210 
mean a temporal and spatial dependent stream of data with incoming video signals from a local camera 222 and 
representing, in the illustrated embodiment, Red, Green and from a remote camera (not shown), and provides the result- 
Blue components of an image, the latter of which may be 65 ing output on a monitor 220. A composite image which 
either static or dynamic. "Video content" may be separated combines the images from the computer 210, the video from 
from the "video envelope" in the "video signal" for the the remote camera, and the video from the local camera 222 
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is then output from the video multiplexing system 213 for co-processor 337 is also referred to as a "scaler" or "scaling 
display on the monitor 220, and the image from the local engine". The co-processor 337 includes multiple user- 
camera 222 is sent to a video conferencing encoder (part of accessible registers (not shown) which can be set with the 
system 213) where it is appropriately encoded before trans- appropriate window size. The co-processor 337 calculates, 
mission to the remote system. 5 using interpolation, additional pixel values for the image 
Configuration and Operation of the Video Multiplexing and/or eliminates pixels from the image in accordance with 
System the window size. According to one embodiment, the video 
FIGS. 3, 4, and 5 illustrate a video multiplexing system co-processor 337 is a Weitck Video Power co-processor 
according to one embodiment of the present invention. Model No. 9130-PFD. However, in alternate embodiments 
During operation, the computer video signal 212 can pass 10 similar co -processors as known to those skilled in the art are 
directly through the video multiplexing system 213 to the substituted. 

video amplifier 316 and the monitor 220. Thus, the video The digital information is then converted to an analog 

multiplexing system passes the computer video signal 212 RGB signal by a video digital to analog (D/A) converter 341 

without modification. Alternatively, the computer video sig- before it is forwarded to the video switch 314. As illustrated 

nal 212 may be combined, in real-time, with video generated is in FIG. 3, the entire foregoing process is accomplished 

from a second video source(s). According to one under the control of a system controller 332 through bus 

embodiment, wherein the multiplexing system of the present arbitration and control logic 345. According to one 

invention is used in a video conferencing environment, the embodiment, the controller 332 is a digital signal processor 

second video source(s) is a local camera 222 and/or a remote (DSP) Model No. 320C80 available from Texas Instruments 

camera. Thus, the motion video signals generated from the 20 of Dallas, Tex. Similarly, the video D/A converter 341 is a 

local and/or remote cameras should operate within the same D/A converter Model No. TVP3026, also available from 

timing and format as the computer video graphics generated Texas Instruments. However, in alternate embodiments 

by the computer 210. Accordingly, in one embodiment the similar DSPs and/or D/A converters as known to those 

incoming motion video signals are converted to analog RGB skilled in the art are substituted. 

signals in order to match the color space of the computer 25 Once the motion video signal(s) is thus converted, it is 

video and the monitor 220, and the timing parameters of the now ready to be directly combined in the switch 314 with the 

computer video signal 212, including horizontal and vertical computer video signal 212 from the computer 210. 

sync and active video time, are analyzed in order to syn- However, before the motion video signals (local and remote) 

chronize the motion video signals with the computer video and computer video signal 212 are combined, the signals are 

signal 212. The multiplexing system 213 accomplishes this 30 synchronized. Synchronization of these signals will now be 

in a manner described in more detail below. discussed. 

I. Motion Video processing II. Synchronization of the Motion Video and Computer 

According to one embodiment of the present invention, Video Signals 

the motion video is provided from a local camera 222 As discussed above, the present invention can be adapted 

(near-side video images) and a remote video camera (far- 35 for use with almost any computer system, including a variety 

side video images), both of which operate according to of display types/formats (e.g., any of the standard VESA 

defined NTSC, PAL or other such standards for video, video formats). In effecting synchronization (in format and 

including receipt of compressed video via a telephone timing) of the motion video signal and the computer video 

interface 431. In order to combine the motion video directly signal, the format of the incoming computer video signal is 

with the video signal 212 from the computer 210, the video 40 initially unknown, and thus the vertical and horizontal 

information contained in the NTSC/PAL-formatted or com- parameters of such signal are unknown. As set forth above, 

pressed motion video signal is converted to an analog RGB such horizontal and vertical parameters are determined in 

format. Thus, the motion video, including video images and order to synchronize the motion video signal and the com- 

other information, generated by both the local video source puter video signal. 

and the remote video source is captured and converted by 45 In order to synchronize the motion video signal with a 

video converter 335a or 3356 to a 4:2:2 YUV formatted data computer video signal of an unknown format, it is important 

stream. If the motion video is either NTSC or PAL that the signals' vertical sync (V ) and horizontal sync 

formatted, it is provided to an NTSC/PAL decoder 335a. (H sy nc) signals, as well as active video time, occur coinci- 

However, if compressed video is received via a telephone dentally. It is further desired to generate a pixel clock for the 

interface 431 then it is instead decoded in an H.320 decoder 50 motion video signal which tracks or is based on the original 

3356. The conversion process of decoders 335a and 3356 is pixel clock of the computer video signal 212, and to use this 

well known to those skilled in the art, and thus will not be generated pixel clock as the basis to create synchronization, 

discussed further except as it pertains to the present inven- This generated pixel clock may or may not be substantially 

tion. identical to the pixel clock of the computer video signal 212, 

The formatted data is output from decoders 335a and 55 based on the "zoom factor", as discussed in more detail 

3356 and placed into a VRAM frame buffer 339 by the video below. The video format analyzer and synchronizer (VFAS) 

co-processor 337. VRAM 339 is preferably a 2 MB VRAM. 324 is used to effect these tasks. 

The video co-processor 337 converts and scales the YUV A. Video Format Analyzer and Synchronizer 

data to digital RGB by reading out the data from the VRAM As seen with reference to FIGS. 3 and 5, portions of the 

339, processing the data to convert same to digital RGB 60 computer video signal 212 (including vertical sync (V^^,), 

data, and writing the thus digitized RGB data back into the horizontal sync (H^J, and the blue component of the 

VRAM 339. The video co -processor 337 scales the YUV computer video signal 212) are provided to the video format 

data in its received format, a 352x288 resolution image in analyzer and synchronizer (VFAS) 324, which analyzes the 

the illustrated video conferencing system, to the appropriate computer video signal 212 to extract video timing informa- 

RGB screen resolution (e.g., 1280x1024) and to the window 65 tion. Basically, the VFAS 324 duplicates the envelope of the 

size in which the motion video is to be displayed through a computer video signal 212 from computer 210, so that the 

process well-known to those skilled in the art. Thus, the signals' envelopes are synchronized. In the illustrated 
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embodiment, VFAS 324 includes a programmable frame 
controller (FC) 311, a video format analyzer (VFA) 318 and 
a wide -band phase locked loop (PLL) 319, although the 
frame controller 311, VFA 318 and PLL 319 may each 
constitute separate devices. A more detailed description of 5 
the operation of the VFAS 324 and its components is 
discussed below. 

1. Programable Frame Controller (FC) 

VFAS 324 includes a programmable frame controller 311. 
The FC 311 includes a series of programmable counters (not to 
shown) and performs typical video frame controller tasks as 
are well known in the art, such as the generation of and 
Vsyno an d me generation of sequential frame buffer memory 
addresses for display purposes. Therefore, the operation of 
FC 311 will not be discussed in further detail except as it 15 
pertains to the present invention. 

FC 311 is used in determining the timing parameters (that 
is, the video envelope) of the output RGB analog video 
signal 322 which will be provided to the monitor 220. All 
timing parameters are thus referenced to the clock signal of 20 
the FC 311. As the frequency of the synthesized pixel clock 
frequently exceeds the frequency capability of many com- 
mon logic devices, the synthesized pixel clock is divided to 
a lower frequency signal. This signal is then used as the FCs 
clock. In one embodiment, the FCs clock is the synthesized 25 
pixel clock (generated by the PLL 319 — discussed below) 
divided by four, which will hereinafter be referred to as the 
Group Clock (GCLK). According to one implementation, 
divider logic available in the video D/A converter 341 is 
used to divide the synthesized pixel clock. However, it is to 30 
be appreciated by those skilled in the art that any similar 
logic capable of scaling frequency may be used. Although 
division is shown in FIG. 5 as occurring outside of the video 
D/A converter, it is to be understood that it has been 
illustrated as such for simplicity and ease of understanding 35 
of the present invention, and not meant as a limitation. 

FC 311 also generates "internal" H^^, and V sync signals, 
which are used internally within the system. Specifically, the 
"internal" V fyMC signal is compared with the V^ MC of the 
computer video signal 212 to determine and maintain ver- 40 
tical phasing. The "internal" V,^ signal is compared with 
the V^ MC of the computer video signal 212 by the VFA 318 
to see if it is coincidental therewith. If it is, then it can be 
concluded that the "internal" V^^ is synchronized with the 
V^„ c of the computer video signal 212 (except for possible 45 
vertical phasing adjustments, as discussed below). The 
"internal" signal is used by the PLL 319 to synthesize 
or generate the original pixel clock from the signal of 
the computer video signal 212 as described below. 

In order to create the internal H^^. signal, the FC 311 50 
divides the GCLK by a programmable integer value called 
"H /ofa/ " and divides that quotient by a programmable value 
referred to as the "zoom factor" to create an internal 
signal, where H^, is the number of group clock periods 
which occur in a horizontal scan line. This internal H 55 
signal is provided to the feedback input of the PLL 319. the 
pixel clock, generated by PLL 319 discussed below, can thus 
be programmably altered by modifying the zoom factor. The 
FC 311 creates the internal V^^. signal by dividing the 
internal horizontal frequency (R^) by V totalJ the latter also 60 
being a programmable integer value equal to the total 
number of horizontal scan lines which occur in a single 
frame. It should be noted that these internal and V^ 
signals are used for synchronization purposes and are sepa- 
rate and distinct from the H jy „ c and V^^. components, 65 
respectively, of the computer video signal 212. For ease and 
clarity of understanding, therefore, the H^,. and V^^. 



components of the computer video signal 212 will herein- 
after be referred to as "external H sync " and "external V jyBC ", 
respectively Accurate determination of H w and V, OM/ are 
thus important in the synchronization of the motion video 
and computer video signals and will be discussed in more 
detail below. 

2. Phase-Locked Loop (PL1S) 

In addition to the FC 311, the VFAS 324 also includes 
PLL 319. While some components of the video envelope of 
the computer video signal 212 may be simply extracted, 
such as H^„ c and V^,. the original pixel clock of the 
computer video signal 212 is not contained therein and is 
synthesized from the computer video signal 212. The pur- 
pose of the PLL 319 is to create (or generate) this pixel clock 
and to maintain synchronization of the FC 311 with the 
computer video signal 212. 

According to one embodiment, this is accomplished 
through the application of the external to the reference 
input of the PLL 319 and the internal H^. from the output 
of FC 311 to the feedback input of the PLL 319. It will thus 
be appreciated that PLL 319 attempts to maintain the feed- 
back frequency (that is, the frequency of internal H^) at 
the same frequency as the reference frequency (that is, the 
frequency of external H sync ). In one implementation, the 
PLL 319 is preferably an ICS 1522 chip available from 
Integrated Circuit Systems, Inc. of Valley Forge, Pa, 

Upon achieving a "synchronized" state, the PLL 319 
output (the synthesized pixel clock) will operate at a fre- 
quency as determined according to the following equation: 

Synthesized pixel clock^^taU^cxternat W^^ZoOm Factor 

Given that the GCLK signal is V4 the frequency of that of the 
synthesized pixel clock, its frequency may thus be described 
as follows: 

^GCLK^tsynthaized pixel 

Thus, it should be noted from the above equation that 
even though the external H^,, frequency is unknown (and 
may indeed never be known), the PLL319 can automatically 
recreate the original pixel clock of the computer video signal 
212 once the correct H, OM/ value is selected. In other words, 
once the correct value for U tota] is determined, it is possible 
to correctly synchronize the external with the internal 
H^. As stated above, the determination of the correct H totaI 
value is described below. 

3. Video Format Analyzer (VFA) 

The VFAS 324 also includes a video format analyzer 318. 
According to one embodiment, the VFA 318 is a pro- 
grammed array logic (PAL) containing a programmable 
counter, although it may constitute any similar logic device 
as known to those skilled in the art, such as an application- 
specific integrated circuit or a field programmable gate array. 
One embodiment of the VFA 318 is illustrated in FIG. 6. 

A primary purpose of the VFA 318 is to analyze the 
computer video signal 212 to determine its video envelope 
characteristics. In most cases, this analysis is performed by 
measuring the time between "events" of interest. "Events" of 
interest involve the video timing parameters of the computer 
video signal 212, and include: the leading edge of external 
Hsyncj the trailing edge of external H_ (ync , the leading edge of 
external V^ w , the trailing edge of external V i>rwc , the leading 
edge of internal V^^, the start of active video (both hori- 
zontal and vertical), and the end of active video (both 
horizontal and vertical). The reference time base for mea- 
surement of these parameters is GCLK (for measurement of 
events less than or equal to a single horizontal scan line) or 
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external (for measurement of events longer than a According to one embodiment of the present invention, 

single scan line). For example, V total (the number of hori- the video conference control application commands the 

zontal scan lines in a frame) is determined by counting in video driver to place a blue border of eight pixels around the 

counter 650, the number of "start of external H^" events screen of monitor 220, prior to the measurement of vertical 

that occur between two successive "start of external V^„ C M 5 resolution by VFA 318, although such border is not needed 

events. until the VFA 318 begins the measurement of the active 

a. Sync Normalization by the VFA video region discussed in this paragraph. The video confer- 
In order for the VFA 318 to commence the various ence control application then sends instructions to the VFA 

measurements of the external H^„ c pulse, it should "know" 318 to commence measurement of the active video time of 

when such pulse is being asserted. The assertion of the io the computer video signal 212 once the blue border is 

external H^. pulse is determined by a "sync normalization" detected. According to one embodiment of the present 

function in normalizer 652 which makes the determination invention, once measurement of the active video region has 

based upon the duty cycle of the pulse. Sync normalization completed, the video conference control application sends a 

is performed because the polarity of the H^ wc and V^ of command over the bi-directional data and control path 215 

incoming computer video signal signals may vary depending 15 to extinguish the blue border. The video conference control 

on the host computer 210. For example, as phase-locked application controls the commencement and substantially 

loops such as PLL 319 operate on the edge of signals, PLL controls the termination of this operation (the multiplexing 

319 should be presented with an pulse of positive system may send interrupt commands to terminate the 

polarity. As seen with reference to FIG. 6, the normalizer operation in certain circumstances, otherwise the blue bor- 

652 accepts either polarity of H sync and V^ and respec- 20 der will be extinguished automatically after a set time 

tively outputs an and V^ pulse of positive polarity. period). Alternatively, the video multiplexing system 213 of 

The normalizer 652 may be implemented in a number of the present invention may instead control the commence - 

ways. One implementation includes a counter which counts ment and termination of this operation. It should also be 

up when a sync signal is positive and down when a sync noted that although the "depth" of the border is eight pixels 

signal is negative; therefore, at the end of the counting 25 in one embodiment, such "depth" may instead constitute a 

period, the sign bit of the counter corresponds to the sign of different number of pixels and may constitute pixels of 

the sync pulse. This sign bit and the original sync pulse are another color (e.g., green or red, or a combination of the 

then provided as inputs to an exclusive-or (XOR) gate, so green, red and blue colors) in alternate embodiments, 

that the output is a positive polarity signal. Alternatively, the Additionally, it is to be appreciated that instead of the 

sync signal may be latched whenever active video is present, 30 automatic extinguishing of the blue border, if desired, the 

and since a sync pulse is not asserted during active video, the appearance of the blue border on the screen of monitor 220 

latched value will correspond to the sync polarity. It will be may be eliminated through known video technique such as 

appreciated that the manner by which the sync pulse is chroma-keying, 
normalized is not limited to these embodiments, and other 

methods of sync nonnalization may be used instead. 35 I Vertical Measurement by the VFA 

b. Active Video Measurements by the VFA _ t_ ,_, . . ■ , . , . 

Again, one of the events of interest which the VFA 318 ° nc / the ^ bo u rder 15 f stablished > the comparator 654 of 

measures is the start and end of active video time. In order FIG k * Slgnals k wl ! en 15 P resent ' ^ the lUustraled 

to measure the active video region of the computer video embodiment the threshold for detection of video by the 

signal 212, the VFA318 measures the vertical front and back 40 c ° m P arator 654 ^25 volts, which is appropriate for most 

"porches" (respectively, V_, _ and V back A as well Vldeo s ^ iGms wh l ch out P ut T lde ? ran ^ from 0 0 t0 0 7 

a^tne horizontal front £d poicta t^tctivdy. V ° l f ^ k Start of actlve vldeo <7" lcal) mea f™ t * 

— fxte 45 -- al rAr d th ; irr r™- line . d 4 whic i h , 

of video (V_ r ) and the time from the leading edge of Vlde ° 1S firSt detec * ed ™ e u end of . aCtl f Vlde0 ^f™ 1 ) 

external V„ to the end of video {\ end ) However, to ensure me f ™ ? P erfo ™ ed by counting the number of hori- 

that the VFA 318 accurately measure! *V stan and V tfm/ , it is ZOntal ?£ c P ulse * ^ hlch °^ ur ^ etween threading ed § e of 

required that the computer video signal contain known video f Xt f ™ al ^ the f m lm u e ^ Vlde ° 15 

on both the first and last scan lines. This is accomplished by 50 aSt d f tci f u In a sumlar mtb ?*> lhe VFA318 can measure 

having a video conference control application or similar the * elwe ? n ev ^ Dts l t ° d f ermine vertical ront porch 

interface program in the computer 210 cause the video vertlcal bat * P. orch ^ vertK ; al ac /™ ^deo-all me asured 

driver (not shown) of monitor 220 to display a border 10 " mts of ho " ZODta u 1 .^ a f P^ 65 < or r horizontal scan lmes) 

consisting of a number of pixels of a specified color at the as ^ i0 } h ™ ^ llled ;° 'he a *- ^ this way the vertical 

boundary of the screen of monitor 220 for identifying the 55 resolullon of the sour ^ of the computer video signal 212 is 

active video region of the computer video signal. The accurate| y determined. 

requirement for the computer 210 to generate known video Horf , Meas , remeQts b the VFA 

content comes from the fact that the VFA 318 requires J 

non-black video content at the borders to measure param- Horizontal measurements are made in a similar manner by 

eters such as front and back porch. However, in many 60 the VFA 318, the difference being that GCLK is used as the 

situations sufficient video content will always exist — such as unit of measurement. The start of horizontal active video 

when miming Windows 95™ with the default blue desktop. measurement is performed by counting in counter 650 the 

In other situations, an enhancement to VFA 318 could be number of GCLK pulses which occur between the leading 

implemented to accumulate video timing information over a edge of the external H^ pulse and the point when active 

period of time with the expectation that the computer 210 65 video is first detected. The end of horizontal active video 

would eventually display video information at the display measurement is performed by counting the number of 

borders. GCLK pules which occur between the leading edge of the 
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external pulse and the point when active video is last quency of the synthesized pixel clock. If the measured 
detected. In addition to the above measurements, the VFA resolution is less than the expected resolution, H fotal is 
318 can measure the interval between events to determine decremented by one, step 734, and step 730 is repeated"! This 
horizontal front porch, horizontal back porch, horizontal has the effect of decreasing the frequency of the synthesized 
active video — again all measured in units of GCLK, as 5 pixel clock. This process is repeated until the measured 
known to those skilled in the art. All of these parameters will horizontal resolution is equal to the expected horizontal 
be used to determine horizontal resolution. resolution. This ensures that the synthesized pixel clock has 

However, as previously discussed, GCLK is a function of bee n correctly locked with the pixel clock of the video signal 
not only the external frequency but of the integer 212. 

parameter H ww/ and the zoom factor (in the FC 311), and 10 In the embodiment shown in FIG. 7, the computer video 
therefore it is important that H be accurately determined signal 212 is then further analyzed to determine if the 
and specified to the FC 311 before any horizontal measure- display of monitor 220 is interlaced or non-interlaced (step 
ments are made. In order to ensure the accuracy of H, ote/ , and 750). This is done in the VFA 318 by consecutive measure- 
thus determined horizontal resolution, this value is not ments of the time difference between the leading edge of 
directly determined by measurement, but instead is deter- 15 V ?MC an d the leading edge of H^. In non-interlaced video, 
mined by use of a lookup table in memory connected to this time difference will be constant. In interlaced video this 
controller 332. Determination of H totaJ will now be dis- difference varies by half a scan line for each successive 
cussed. Ysync- l ^ interlaced video is detected, the condition is flagged 

Measurement of W toud and Video Synchronization by the anc * operation may be aborted, step 755, if desired, or the 
Video Fornat Analyzer and Synchronizer (VFAS) 20 process may continue and the remaining video timing 

FIG. 7 is a flowchart illustrating the steps associated with parameters may be next measured and adjusted, step 760. 
establishing video synchronization between the motion Once determination is made as to whether the display is 
video signals and the computer video signal 212 according interlaced or not, the remaining video timing parameters are 
to one embodiment of the present invention. It is to be measured and adjusted accordingly in a manner similar to 
appreciated, however, that FIG. 7 illustrates only one 25 tnat discussed above, step 760. More specifically, the VFA 
embodiment, and that the order of the steps may be altered 318 measures the remaining horizontal and vertical param- 
and/or steps removed. eters and programs the FC 311 in accordance with these 

First, the VFA 318 measures the vertical resolution of the measurements. If desired, the video conference control 
computer, step 710. As discussed above, the video confer- application can be caused to extinguish the blue border, 
ence control application running on computer 210 causes the 30 At this point, the motion video is now synchronized with 
video driver in computer 210 to display a blue border on the tne computer video signal 212 in all ways except for an 
screen of the monitor 220. Using this blue border, the VFA expected phase shift in vertical sync. Thus, the next step, 
318 measures the vertical active video which is the vertical ste P 770, is to adjust the vertical phase of the motion video 
resolution of the computer video signal 212. signal in order to match the vertical phase of the computer 

After this measurement, the H MM/ must be determined. 35 video signal 212. The VFA 318 accomplishes this by count- 
ing in counter 650 the U sync pulses between the leading edge 
of the internal pulse and the leading edge of the V 5ywc 
pulse of the computer video signal 212. This count repre- 
sents the phase difference between the motion video signal 
40 and the computer video signal 212 in units of horizontal scan 
lines. This phase difference is corrected by increasing V tota} 
by the phase difference for a single frame. V Mtaf is then 
immediately restored. The two video signals should now be 
completely synchronized. 
45 Another important feature of the VFA is that if vertical 
lock between the motion video and computer video signals 
is lost (that is, the internal V syftc is not synchronized with the 
external V jync ), interrupt logic 656 of FIG. 6, under the 
control of control logic 658, will generate an interrupt. The 
50 VFA 318 can be programmed to prevent multiplexing of the 
motion video and computer video signals if synchronization 
is lost. In such a case, display of motion video on monitor 
220 of computer 210 will be blanked until vertical lock is 
re-established. Likewise, multiplexing will occur only when 
55 vertical lock is re-established. Alternatively, if desired, VFA 
318 can be programmed to multiplex the motion video and 
computer video signals irrespective of whether vertical lock 
is maintained. The latter is useful, for example, when trying 
to re-establish lock during video conferencing without los- 
Once the estimated H totai has been calculated, the VFA 60 ing the video conferencing channel altogether in the process, 
318 measures the horizontal resolution of the computer or if the display resolution is changed on the fly. 
video signal 212 by again measuring the interval between Horizontal Screen Resolution Modification 
events of interest, step 730. The measured resolution is then As discussed above, a separate frame buffer memory 
compared to the expected resolution by VFA 318, step 740, (VRAM 339 of FIG. 3) is used for the storage of video 
and if the measured resolution is less than the expected 65 information for the motion video. In the illustrated 
resolution, U totaJ is incremented by one, step 732, and step embodiment, VRAM 339 is a 2 MB memory, with 1.5 MB 
730 is repeated. This has the effect of increasing the fre- being available for the display of motion video (the remain- 



Again, this value will be used to synthesize an internal H 
and the synthesized pixel clock. In step 720, the controller 
332 uses the thus determined vertical resolution to look up 
the corresponding horizontal resolution according to Table I 
below which is stored in memory as discussed above, in 
order to determine U total . In alternate embodiments, other 
methods for determining horizontal resolution are used 
rather than a lookup table, such as calculating the horizontal 
resolution using equations well known in the art based on the 
vertical resolution. 



TABLE I 



Measured Vertical Resolution 


Expected Horizontal Resolution 


480 to 599 


640 


600 to 767 


800 


768 to 863 


1024 


864 to 1023 


1152 


1024 to 1199 


1280 


1200 and above 


1600 



A value for H total is then estimated by the following formula: 
Hid*/ (est.)-1.2x (Expected Horizontal Resolution /4) 
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ing 0.5 MB being used for the storage of various control 
information and received images). Having 1.5 MB available 
for the motion video display can result in storage space 
problems for large screen resolutions. The amount of 
memory required to store an image can be identified by the 5 
following equation: 

mcmory-H^^xV^^^xcolor depth (in bytes) 

Thus, a 1024x768 screen resolution with 16 bits of color 
requires 1024x768x2 bytes, which is 1,572,864 bytes, or 1.5 JO 
MB. Similarly, a 1280x1024 screen resolution with 16 bits 
of color requires 1280x1024x2 bytes, which is 2,621,440 
bytes, or 2.5 MB. Thus, a screen resolution of 1280x1024 
cannot be conventionally supported by a 1.5 MB memory 
space. is 

The present invention resolves this problem by altering 
the width of the pixels (either enlarging the width or 
reducing the width) in the motion video, a process referred 
to as "zooming". By enlarging the pixel widths and corre- 
spondingly reducing the horizontal resolution, the present 20 
invention is able to reduce the amount of memory space 
necessary to store the image. Thus, by way of example, if the 
screen resolution is 1280x1024 with 16 bits of color, the 
present invention alters the widths of the pixels so that at 
most 768 pixels per scan line are displayed. The effective 25 
screen resolution then becomes 768 (or less)xl024, which 
requires no more than 768x1024x2 bytes, which is 1,572, 
864 bytes, or 1.5 MB. 

According to one embodiment of the present invention, 
the pixels are modified only in the horizontal dimension. 30 
That is, the effective horizontal screen resolution may be 
modified, however the vertical screen resolution is not 
altered. 

According to one embodiment, the present invention is 
implemented in a combination of hardware and software. 35 
The hardware components of overlaying motion video on an 
analog signal are discussed above with reference to FIGS. 
3-7. The software portion of the present invention is illus- 
trated in FIG. 8, which illustrates a storage medium 830 
including a zoom control portion 810 and a motion video 40 
overlay control portion 820. The motion video overlay 
control portion 820 controls the setting of various param- 
eters of the hardware components, as discussed above. The 
zoom control portion 810 controls the identification of a 
correct zoom factor and the setting of the hardware compo- 45 
nents based on this zoom factor, as discussed in more detail 
below. The storage medium 830 is meant to represent a wide 
variety of machine-readable media in which the present 
invention can be implemented, including conventional stor- 
age devices (such as a floppy disk, random access memory, 50 
read only memory, Flash memory, etc.), as well as discrete 
hardware or firmware. In the illustrated embodiment, control 
portions 810 and 820 are implemented as a series of software 
routines executed by the video multiplexing system 213 of 
FIG. 2. These software routines comprise a plurality or 55 
series of instructions to be executed by system controller 
332 of FIG. 3. 

FIG. 9 is a flowchart illustrating the steps followed in 
modifying a horizontal screen resolution according to one 
embodiment of the present invention. As illustrated, the 60 
zoom control logic initially identifies the current video 
signal resolution, step 905. This video signal resolution is 
the resolution of the video signal 212 of FIG. 3. The zoom 
control logic then checks whether the identified video signal 
resolution is supported by the available memory (1.5 MB in 65 
the illustrated embodiment), step 910. If there is sufficient 
memory for the identified video signal resolution, then the 
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zoom control logic creates a "magnification" table, step 915. 
The magnification table will be used to increase the hori- 
zontal screen resolution as discussed in more detail below. 
However, if there is insufficient memory for the identified 
video signal resolution, then the zoom control logic creates 
a "zoom" table, step 917. The zoom tabic will be used to 
decrease the horizontal screen resolution as discussed in 
more detail below. The zoom control logic then determines 
a new (modified) horizontal resolution, using either the 
magnification table from step 915 or the zoom table from 
step 917, based on the available memory, step 920, and 
programs the PLL 319 based on the new horizontal 
resolution, step 925. This programming of the PLL 319 
alters the pixel width for the motion video to be different 
than that used for the display of images from the computer 
system 210. 

FIGS. 10 and 11 illustrate the pixel width modification 
according to one embodiment of the present invention. FIG. 
10 illustrates a screen 1000 including an input scan line A 
having a horizontal resolution of 1280 pixels (e.g., from 
computer system 210), a concurrent scan line B having a 
horizontal resolution of 768 pixels (e.g., zoomed motion 
video), and a composite scan line C having two different 
horizontal resolutions (e.g., a combination of both the 
zoomed motion video and the video signal from computer 
system 210). 

As illustrated, the pixels of the concurrent scan lines A, B, 
and C "line up" with each other at both the start of the line 
(the beginning of pixel 1 in each scan line) and at the end of 
the line (the end of pixel 1280 in scan lines A and C and pixel 
768 in scan line B). However, each pixel in the scan line B 
is wider than the pixels in the scan line A. By widening the 
pixels in scan line B, fewer pixels are displayed per scan 
line, and thus memory requirements for the display of the 
scan line are reduced. 

Additionally, the first four pixels and at least the last four 
pixels of scan line C (e.g., from the computer system 210) 
are the same size as those in scan line A, while several of the 
pixels in-between (e.g., from the local camera 222) are the 
same size as those in scan line B. Scan line C illustrates a 
composite scan line of PC video and motion video. 

The pixel widths of a scan line are modified by altering 
the synthesized pixel clock, as illustrated in FIG. 11. An 
Hsync signal 1110 is illustrated which indicates the beginning 
and ending of a horizontal scan line. The H sync signal 1110 
is provided as part of the video signal 212 from the computer 
system 210. An active video line 1120 is illustrated which 
indicates when active video is being displayed, either 
images from the computer system 210 or the motion video. 
Video is being displayed when the active video line 1120 is 
"high" (that is, from time t 4 to time U)). It should be noted 
that the active video line 1120 is included in FIG. 11 to assist 
in the explanation of the present invention; the active video 
line 1120 is not an actual signal input to or generated by the 
video multiplexing system of the present invention. The time 
between the end of H^, time t^, and the start of the active 
video, time t 4 , is referred to as the "back porch". Similarly, 
the time between the end of the active video, time tg, and the 
start of the next H^„ c time t 30 , is referred to as the "front 
porch". 

The display of pixels is dependent on a pixel clock. Anew 
pixel is displayed at each rising edge of the pixel clock. 
Thus, the number of pixel clock periods in the horizontal 
scan line corresponds to the number of pixels which are 
displayed on that scan line. A pixel clock 1130 is illustrated 
as well as a synthesized pixel clock 1140. The pixel clock 
1130 is used for the display of pixels of images from video 
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signal 212, while the synthesized pixel clock 1140, gener- 
ated by PLL 319 of FIG. 3 discussed above, is used for the 
display of pixels of images from the motion video. As 
illustrated, a new pixel on the horizontal scan line is gen- 
erated at times t 4 , t 5 , and t 7 for images from analog RGB 
signal 212, while a new pixel on the horizontal scan line is 
generated at times t 3 , t 6 , and t 8 for the motion video. 

As discussed in more detail below, the video switch 314 
of FIG. 3 outputs video information from one of either video 
signal 212 or RGB signals from D/A converter 341 
(corresponding to the motion video). If the video informa- 
tion is output from video signal 212, then the pixels are 
displayed in accordance with pixel clock 1130, whereas if 
the video information is output from D/A converter 341, 
then the pixels are displayed in accordance with synthesized 
pixel clock 1140. 

In the illustrated embodiment, the video switch 31 zl can 
switch its output between the two source signals, video 
signal 212, and the RGB signals from D/A converter 341 at 
the beginning of every GCLK. Thus, it is preferred that the 
rising edge of the synthesized pixel clock 1140 line up 
closely with the rising edge of the pixel clock 1130 at the 
point where a switch from one source to another occurs, 

FIG. 12 is a flowchart illustrating the steps followed in 
identifying a proper horizontal resolution, based on avail- 
able memory, according to one embodiment of the present 
invention. The proper value for H total from the computer 
system 210 (PC H foM/ ) is first determined, step 1205, in 
accordance with the processes discussed above. The zoom 
control logic then determines a "modified" W total based upon 
pixels from the PC's point of view, step 1210. The modified 
H, OM/ value is initialized to be the same as H toral , and is 
subsequently decremented by four for each iteration through 
step 1210. 

A zoom table entry is then calculated based on the 
modified H WCfl „ step 1215. A zoom table entry includes at 
least four components: the modified H MM/ , the zoom factor, 
the zoomed horizontal resolution, and the alignment error. 
The modified H total is the value determined in step 1210. The 
zoom factor is calculated by dividing U total by the modified 
H Wta/ . The zoomed horizontal resolution (Z_HRES) is 
calculated by dividing the PC horizontal resolution by the 
zoom factor. The alignment error is calculated according to 
the following equation: 

errorazoom factorx(Z_HRES-(4.0x truncate ((Z„HRES/4.0)+ 
0-5)))) 

The zoom control logic then checks whether a sufficient 
number of zoom table entries have been calculated, step 
1220. According to one embodiment, the zoom control logic 
calculates 512 entries. However, alternate embodiments can 
generate a lesser or greater number of entries. Table II below 
illustrates a sample zoom table corresponding to a PC 
resolution of 1280x1024 and a PC H iotal of 1728. For ease 
of explanation, only a few entries have been included in 
Table II rather than 512. 

TABLE II 



Zoomed 
Horizontal 
Resolution 



Zoom 
Factor 



Alignment 
Error 



Modified 
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TABLE Il-continued 



Zoomed 
Horizontal 



Zoom Alignment Modified 



Resolution 


Factor 


Error 




Htoial 


1274.074 


1.004651 


-1.9 


1720 


430 


770.3704 


1.661538 


-2.7 


1040 


260 


767.4074 


1.667954 


-1.0 


1036 


259 


764.4444 


1.674419 


0.7 


1032 


258 


761.4815 


1.6S0934 


2.5 


1028 


257 


758.5185 


1.6875 


-2.5 


1024 


256 



20 



25 



30 



35 



40 



45 



50 



55 



1280 
1277,037 



1 

1.00232 



0.0 
1.0 



1728 
1724 



432 
431 



65 



Once the zoom table is created, the zoom control logic 
uses the zoom table to identify a first zoomed resolution 
which fits into available memory, step 1225. This first 
zoomed resolution is the largest zoomed horizontal resolu- 
tion that can be used with the available memory. In the 
illustrated embodiment, with 16 bits of color (2 bytes), the 
first zoomed horizontal resolution (Z_HRES) is determined 
using the following calculation: 

Available Memory ^Z_HRESx Vertical Resolutionx2 

When the largest value of zoomed horizontal resolution 
(Z_HRES) in the table which satisfies the above calculation 
is identified, the zoom control logic evaluates a number of 
subsequent entries in the table, with lower zoomed horizon- 
tal resolutions, to identify a second zoomed resolution which 
provides a small alignment error, step 1230. This alignment 
error indicates the alignment difference of visible pixels 
between is internally generated video (from VRAM 339) 
and external generated video (from computer system 210). 
The zoom control logic searches for an alignment error 
within a predetermined range, (e.g., between -0.3 and 0.3). 
In the illustrated embodiment, the zoom control logic only 
looks at a few zoomed horizontal resolutions because each 
subsequent entry represents a loss of resolution. In the 
illustrated embodiment, the zoom control logic looks at a 
number of subsequent table entries, referred to as the "search 
range", identified by the following calculation: 

search length-alignment error at first Z_HRESx4 

The zoom control logic uses the largest zoomed horizontal 
resolution within the search range with an alignment error 
within the predetermined range as the second zoomed reso- 
lution. If no alignment error is within the predetermined 
range, then the zoom control uses the zoomed horizontal 
resolution within the search range with the smallest align- 
ment error as the second zoomed resolution. It is to be 
appreciated that the second zoomed horizontal resolution in 
step 1230 may be the same as the first zoomed horizontal 
resolution in step 1225. 

Generation of a magnification table (step 915 of FIG. 9) 
rather than a zoom table is done in an analogous manner, 
except that the magnification table entries are calculated by 
incrementing the modified H toW/ value by four for each 
iteration through step 1210. Thus, zoom factors in the 
magnification table have a value between 0 and 1.0. 'llie 
zoom control logic selects a first and a second zoomed 
horizontal resolution from the magnification table in the 
same maimer as it selects a first and second zoomed hori- 
zontal resolution from the zoom table. 

The zoom control logic then programs the frame control- 
ler 311 with this second zoomed horizontal resolution (from 
either the magnification table or the zoom table), step 1235. 
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According to one implementation, frame controller 311 is then a window area with dimensions of 320x240 would be 

programmed with integer values, so the second zoomed available for the image. However, since the image has a 

horizontal resolution is thus rounded to the closest integer dimension of 352x288, then 32 pixels in the horizontal 

value. Thus, the frame controller will display the motion dimension and 48 pixels in the vertical dimension are "lost" 

video with a horizontal resolution equal to this second 5 by the scaling of co-processor 337. In other words, the 

zoomed horizontal resolution identified in step 1230, scaling of co-processor 337 modifies the image to eliminate 

Thus, by way of example, referring to Table II with a color 32 pixels in the horizontal dimension and 48 pixels in the 

depth of 16 bits, the first zoomed horizontal resolution which vertical dimension. However, "magnifying" the motion 

fits into the available 1.5 MB of memory is 767.4074. The video according to the present invention with a screen 

zoom control logic then searches the next 4 entries and 10 resolution of 1638x480, that same image being displayed on 

identifies the zoomed horizontal resolutions of 764.4444, one-quarter of a screen would have a dimension of 819x240. 

which has an alignment error of 0.7, as the second zoomed Thus, although 48 pixels are still lost in the vertical 

resolution. The value of 764.4444 rounds to an integer value dimension, no pixels are lost ill the horizontal dimension, 

of 764, so 764 is thus programmed into the frame controller Therefore, the quality of the image has been improved due 

311. 15 to the co-processor 337 having to remove fewer pixels. 

According to one embodiment of the present invention, Thus, particularly when used with a traditional video 
the motion video which is overlaid onto the video signal 212 conferencing system, the present invention advantageously 
is received as a Common Imaging Format (OF) image. The reduces the memory requirements for high screen resolu- 
CIF standard uses an image dimension of 352x288, which is tions without sacrificing image quality, and at the same time 
a common image resolution for current video conferencing 20 can increase image quality for lower screen resolutions, 
applications. In a conventional video system, a scaling It should also be noted that although the above discus- 
engine, such as co-processor 337 of FIG. 3 described above, sions describe the horizontal resolution modification as 
is used to scale a lower resolution image to a higher being preformed for an overlapping motion video, the 
resolution image. Such scaling typically involves the use of present invention is not limited to such embodiments. In 
interpolation, in which the scaling engine calculates addi- 25 alternate embodiments, the present invention is imple- 
tional pixel values for images based on the original image's mented on "full screen" motion video, 
pixel values. By way of example, a 352x288 resolution Smart Video Switch and Multiplexer Control Mask 
image can be scaled to a 1280x1024 resolution image by Once the motion video is processed appropriately, the 
calculating 928 new pixel values for each horizontal line and aforementioned horizontal and vertical parameters of the 
738 new pixel values for each vertical column. Such scaling 30 display of the computer 210 determined, the zoom factor 
techniques are well known to those skilled in the art and thus determined, and the motion video and computer video signal 
will not be discussed further except as they pertain to the 212 synchronized, the video multiplexing system 213 deter- 
present invention. mines whether the computer video signal 212 or the motion 

Thus, it is to be appreciated that the use of the present signal is to be displayed on the monitor 220. 
invention in a traditional video conferencing system does 35 Referring again to FIG. 3, the smart video switch 314 is 
not affect the image quality of the motion video, despite the an analog switch used to source either the computer video 
fact that the zoomed horizontal resolution of the CIF images signal 212 or the motion video output from the D/A con- 
may in fact be less than the resolution of the video signal verier 341 to the screen of computer monitor 220 through 
212. At full screen size, the zoomed horizontal resolution video amplifier 316. In operation, the smart video switch 
(e.g., 768x1024), although less than the video signal 212 40 314 combines, or multiplexes, the computer video signal 
resolution (e.g., 1280x1024) is still greater than the original with the motion video output from the video D/A converter 
image (e.g., 352x288), so no image quality has been lost. 341, and then, based upon information generated by a 

In the discussions above, the identification of the zoomed multiplexer control mask (MCM) 334, selects the correct 

horizontal resolution to be used by the present invention is image source (computer 210 or motion video source) for 

performed by generating a zoom or magnification table. 45 display on the screen of monitor 220. According to one 

However, it is to be appreciated that alternate embodiments implementation, MCM 334 is a 16 k byte static random 

need not generate such tables — by way of example, other access memory (SRAM), while the smart video switch 314 

calculations may be used, or the table entries for selected is a QS4A205 high speed analog multiplexer available from 

common resolutions may be pre-coded into a table (thereby Quality Semiconductor, Inc. of Santa Clara, Calif. However, 

eliminating an "on the fly" calculation). It is also to be 50 alternate embodiments can use similar commercially avail - 

appreciated that alternate embodiments can combine the able memories and/or multiplexers, 
zoom and magnification table into a single table rather than In the present invention, the smart video switch 314 is 

two separate tables. capable of switching between images on a boundary of 

When zoom factors ranging between 0.0 and 1.0 are used, every four pixels, but may alternatively be made to switch 

the synthesized pixel clock 1140 of FIG. 11 above has a 55 on smaller boundaries (e.g., every two pixels or every pixel), 

shorter clock period than that of pixel clock 1130. By way While switching on smaller boundaries would increase the 

of example, the image resolution of video signal 212 may be resolution of the border between the computer video signal 

640x480. Using a 640x480 screen resolution for the motion 212 from the host computer 210 and the video image to be 

video only requires 6 14,400 bytes (assuming 2 bytes of color displayed, it would also increase memory requirements of 

per pixel), or approximately 0.6 MB. Thus, an additional 60 the MCM 334, and thus such increase in switch 314 switch - 

approximately 0.9 MB of the VRAM 339 is available for ing speed depends on the amount of memory available to the 

display. Thus, the original screen resolution of 640x480 can MCM 334. 

be magnified to 1638x480 with a zoom factor of 0.39. As each raster line of the combined video, motion video 

Such an increase in screen magnification can increase the and computer video signal, is formed the system of the 

image quality of the motion video. For example, if a 65 present invention determines which image source is to be 

352x288 image were to be displayed in a window one- displayed — the computer video signal 212 or the motion 

quarter the size of a screen with a resolution of 640x480, video signal output from video D/A converter 341. This 
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decision is controlled by the MCM 334. Under the control 
of controller 332, the MCM 334 controls the smart video 
switch 314 to output one of either the computer video signal 
212 from the computer 210 or motion video signal output 
from the video D/A converter 341. The "mask" serves as a 
template for drawing a background "window" into which the 
motion video signal from the local video source or remote 
video source will ultimately be displayed. Thus, the MCM 
334 is used to determine the display size and location of the 
window(s) which will display the video signal received from 
the remote and/or local video signal sources. 

More particularly, each bit in the MCM 334 controls the 
source of video for a pixel or multiple pixel block (e.g., a 
single bit in the MCM 334 controls a 4x4 pixel array in one 
implementation). The MCM 334 generates a continuous bit 
stream of pixel data having values of "0" or "1". Such values 
represent whether the switch 314 will source the computer 
video signal 212 from the computer 210 or whether it will 
source the motion video signal output from video D/A 
convert 341. According to one embodiment, a bit value of 
"0" will cause the smart video switch 314 to source the 
computer video signal 212 for four horizontally adjacent 
pixels of the screen, while a bit value of "1" will cause the 
smart video switch 314 to source the motion video signal 
output from the video D/A converter 341 for the four pixels 
for four scan lines. Again, resolution can be increased, if 
desired (e.g., in an alternate implementation a single bit in 
MCM 334 controls the source of video for a single pixel). 
The "windows" created by MCM 334 are much like other 
computer "windows" known in the art. Again, local video 
from the local video source 222 is ultimately displayed in 
one "window" (e.g., window 104 of FIG. 1), while video 
from the remote video source is ultimately displayed in 
another window (e.g., window 102 of FIG. 1). Information 
generated by the computer video signal 212 is displayed in 
another area, such as window 106 of FIG. 1. Additional 
"windows" may be displayed by the MCM 334. According 
to one embodiment, these windows are used to display 
graphics information from an application such as user inter- 
face program and which is used for controlling the system of 
the present invention. It is to be appreciated that the MCM 
334 may be programmed to display windows in rectangles 
or various other shapes. For example, if the granularity of 
the MCM is made to be very fine (e.g., a resolution of one 
or two pixels), it is possible to create a window which has 
an arbitrary shape. Thus, an oudine view of the speaker 
could in fact be the "window". 

The content of the memory of MCM 334 may be con- 
trolled by the video conference control application or similar 
user interface program running in the computer 210. In the 
video conferencing embodiment, this application would 
control the size, position and stacking order of the near- and 
far-side windows. The user controls these factors in the same 
manner he or she would for any computer window. For 
example, if the user resizes or moves a window containing 
the near- or far-side motion video, this information is 
transmitted from the computer system 210 to the multiplexer 
system of the present invention through controller 332 and 
bus arbitration and control logic 345, the system controller 
332 adjusting the contents of the MCM 334 memory accord- 
ingly. 

Although the multiplexing system of the present inven- 
tion has been discussed in reference to a video conferencing 
environment, it will be appreciated that such multiplexing 
system has a broader range of uses, and may be used in 
several applications, such as a communications broadcast 
environment, a multimedia environment, a DVD-recorder 
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and/or playback application, other computer and video 
applications, etc. Additionally, such multiplexing system 
may be used in a video test instrument or in a video format 
converter. Furthermore, although the source of the computer 

5 video signal 212 is described above as a computer outputting 
VGA-style RGB video with separate H^,, and V^ pulses, 
the present invention is not so limited, and may be extended 
to all versions of RGB video, including video with compos- 
ite Hjync and V ^ video with sync on green, as well as RS 

10 170 compliant video and its derivatives. In addition, 
although control of the motion video frame controller 311 is 
effected using software commands, it may alternatively be 
effected using a hardware implementation. 
According to an alternate embodiment of the present 

is invention, multiplexing system 213 provides memory man- 
agement based on window sizes and positions. In this 
alternate embodiment, rather than having the entire screen 
which is zoomed by the present invention, a region (e.g., a 
single rectangular region) of the display is defined which 

20 encompasses all of the windows containing motion video. 
The resolution of this region is used to compute the proper 
zoom factor rather than the resolution of the entire screen. 
This region is then considered to be the motion video and is 
zoomed as discussed above. According to one implementa- 

25 tion this region is dynamically defined and is modified each 
time a window within the region is closed or re-sized, or a 
new window on the display device is opened (e.g., either a 
newly created window or a previously minimized window 
being re-stored to its previous size). 

30 FIG. 13 illustrates a display according to One such 
alternate embodiment. The display 1300 includes two win- 
dows 1301 and 1302. A region 1305 is defined by the zoom 
control logic which includes both of the windows 1301 and 
1302. As illustrated, although the computer system is using 

35 a screen resolution of 1280x1024, only a 900x600 region 
includes windows being displayed. Thus, the zoom control 
logic calculates a zoom factor based on a computer screen 
resolution of 900x600 rather than 1280x1024. Thus, a zoom 
factor of less than 1 could be used even though the computer 

40 resolution suggests a zoom factor greater than 1 due to the 
implied requirement of 2.5 MB (1280x1024x2 bytes). 

Furthermore, although the multiplexing system of the 
present invention has been discussed in reference to receiv- 
ing motion video from a camera, any of a wide variety of 

45 devices can be the source of such motion video. Examples 
of such sources include a video cassette player, laser disk 
player, television, digital versatile disk (DVD) player, etc. 

In addition, although the multiplexing system of the 
present invention has been discussed in reference to receiv- 

50 ing a motion video signal, the present invention is not 
limited to such embodiments. In alternate embodiments 
other types of signals are received, such as "static" or "still" 
images (e.g., one or more frames of the same image), from 
any of a wide variety of conventional sources. By way of 

55 example, VRAM 339 of FIG, 3 may repeatedly provide the 
same data to D/A converter 341 for multiple frames of 
computer video signal 212. By way of another example, the 
same image may repeatedly be input to VRAM 339. It is to 
be appreciated that the zooming of the present invention is 

60 not dependent on the source of the video signal or the video 
content (e.g., motion video or otherwise). 

Additionally, although the zooming of the present inven- 
tion has been discussed in reference to a video multiplexing 
system, the present invention is not limited to such embodi- 

65 ments. In alternate embodiments, the zoomed motion video 
encompasses the entire display of monitor 220 rather than 
being an overlay. Thus, it is to be appreciated that in such 
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alternate embodiments much of the hardware of multiplex- 
ing system 213 for synchronizing the motion video to the 
computer video signal is not necessary, as there are not two 
different signals to be synchronized. 

Furthermore, in the discussions above, the multiplexing 
system 213 is described as being separate from the computer 
system 210. It is to be appreciated that, in alternate 
embodiments, multiplexing system 213 can be incorporated 
into computer system 210. 

Thus, a method and apparatus for modifying screen 
resolution based on available memory has been described. 
Whereas many alterations and modifications of the present 
invention will be comprehended by a person skilled in the art 
after having read the foregoing description, it is to be 
understood that the particular embodiments shown and 
described by way of illustration are in no way intended to be 
considered limiting. References to details of particular 
embodiments are not intended to limit the scope of the 
claims. 

What is claimed is: 

1. A method comprising the steps of: 

identifying a first screen resolution for a display, the first 
screen resolution corresponding to a resolution setting 
for the display; 

receiving a video signal to be displayed on the display 
having a video signal resolution; and 

creating a second screen resolution for displaying the 
received video signal on the display based on the video 
signal resolution, the first screen resolution and a 
comparison of the second screen resolution to the 
amount of memory available in a frame buffer for the 
display, to create the largest second screen resolution 
that can be used with the memory available. 

2. The method of claim 1, wherein the step of creating 
comprises the step of altering a pixel width for each of a 
plurality of pixels of the received video signal. 

3. The method of claim 2, wherein the step of creating 
further comprises the steps of: 

identifying a plurality of screen resolutions which can be 
used as the second screen resolution; and 

selecting as the second screen resolution the one of the 
plurality of screen resolutions which provides a small- 
est alignment error with the first screen resolution. 

4. The method of claim 2, wherein the step of altering a 
pixel width comprises the step of altering a pixel clock 
period corresponding to the second screen resolution. 

5. The method of claim 4, wherein the step of altering the 
pixel clock period comprises the step of increasing the pixel 
clock period. 

6. The method of claim 2, wherein the plurality of pixels 
comprises every pixel to be displayed as part of the image. 

7. The method of claim 1, wherein displaying the received 
video signal on the display comprises displaying the 
received video signal on the entire display. 

8. The method of claim 1, further comprising the step of 
overlaying an image having the second screen resolution on 
an image having the first screen resolution. 

9. The method of claim 1, wherein the received video 
signal includes one or more windows which provide video 
information from one or more video conferencing cameras. 

10. The method of claim 1, wherein a horizontal resolu- 
tion of the second screen resolution is less than a horizontal 
resolution of the first screen resolution. 

11. The method of claim 1, wherein a horizontal resolu- 
tion of the second screen resolution is greater than a hori- 
zontal resolution of the first screen resolution. 

12. The method of claim 1, wherein the step of creating 
the second screen resolution further comprises generating a 
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first candidate resolution and a second candidate resolution, 
comparing each candidate resolution to the available frame 
buffer memory, selecting a candidate resolution based on the 
comparison. 

5 13. The method of claim 12, wherein the step of creating 
the second screen resolution further comprises generating 
further candidate resolutions by incrementing the resolution 
by a selected amount. 

14. The method of claim 12, wherein the step of selecting 
JQ a candidate further comprises selecting the largest resolution 

that will fit into the available frame buffer memory. 

15. The method of claim 1, wherein the step of creating 
the second screen resolution further comprises generating a 
first candidate resolution and a second candidate resolution, 
determining the alignment error between each candidate 

15 resolution and the first resolution, comparing each candidate 
resolution to the available frame buffer memory, and select- 
ing a candidate resolution with the largest resolution that 
will fit into the available frame buffer memory from among 
those candidate resolutions that have an alignment error less 

20 than a desired threshold. 

16. A machine-readable medium having stored thereon a 
plurality of instructions, designed to be executed by a 
processor, for implementing a function for identifying a first 
screen resolution for a display, the first screen resolution 

25 corresponding to a resolution setting for the display, for 
receiving a video signal to be displayed on the display 
having a video signal resolution, and for creating a second 
screen resolution for displaying the received video signal on 
the display based on the video signal resolution, the first 

30 screen resolution and a comparison of the second screen 
resolution to the amount of memory available in a frame 
buffer for the display, to create the largest second screen 
resolution that can be used with the memory available. 

17. The machine-readable medium of claim 16, wherein 
35 the plurality of instructions for implementing a function for 

creating comprises a plurality of instructions for implement- 
ing a function for altering a pixel width for each of a 
plurality of pixels of the received video signal. 

18. The machine-readable medium of claim 17, wherein 
40 the plurality of instructions for implementing a function for 

creating are further for implementing a function for identi- 
fying a plurality of screen resolutions which can be used as 
the second screen resolution, and for selecting as the second 
screen resolution the one of the plurality of screen resolu- 
45 tions which provides a smallest alignment error with the first 
screen resolution. 

19. The machine-readable medium of claim 17, wherein 
the plurality of instructions for implementing a function for 
altering a pixel width comprises a plurality of instructions 

50 for implementing a function for altering a pixel clock period 
corresponding to the second screen resolution. 

20. The machine-readable medium of claim 19, wherein 
the plurality of instructions for implementing a function for 
altering the pixel clock period comprises plurality of instruc- 

55 tions for implementing a function for increasing the pixel 
clock period. 

21. The machine-readable medium of claim 17, wherein 
the plurality of pixels comprises every pixel to be displayed 
as part of the image. 

60 22. The machine-readable medium of claim 16, wherein 
displaying the received video signal on the display com- 
prises displaying the received video signal on the entire 
display. 

23. The machine-readable medium of claim 16, wherein 
65 the plurality of instructions are further for implementing a 
function for overlaying an image having the second screen 
resolution on an image having the first screen resolution. 



01/29/2002, EAST Version: 1.02.0008 



US 6,313,822 Bl 



23 



24 



JO 



20 



24. The machine-readable medium of claim 16, wherein 
the received video signal includes one or more windows 
which provide video information from one or more video 
conferencing cameras. 

25. The machine-readable medium of claim 16, wherein 
a horizontal resolution of the second screen resolution is less 
than a horizontal resolution of the first screen resolution. 

26. Hie machine-readable medium of claim 16, wherein 
a horizontal resolution of the second screen resolution is 
greater than a horizontal resolution of the first screen reso- 
lution. 

27. The machine-readable medium of claim 16, wherein 
the plurality of instructions for implementing a function for 
creating comprises a plurality of instructions for implement- 
ing a function for generating a first candidate resolution and 
a second candidate resolution, comparing each candidate 
resolution to the available frame buffer memory, selecting a 
candidate resolution based on the comparison. 

28. The machine-readable medium of claim 27, wherein 
the plurality of instructions for implementing a function for 
creating comprises a plurality of instructions for implement- 
ing a function for generating further candidate resolutions by 
incrementing the resolution by a selected amount. 

29. The machine-readable medium of claim 27, wherein 
the plurality of instructions for implementing a function for 
selecting a candidate comprises a plurality of instructions for 
implementing a function for selecting the largest resolution 
that will fit into the available frame buffer memory. 

30. The machine-readable medium of claim 16, wherein 
the plurality of instructions for implementing a function for 
creating comprises a plurality of instructions for implement- 
ing a function for generating a first candidate resolution and 
a second candidate resolution, determining the alignment 
error between each candidate resolution and the first 
resolution, comparing each candidate resolution to the avail- 
able frame buffer memory, and selecting a candidate reso- 
lution with the largest resolution that will fit into the 
available frame buffer memory from among those candidate 
resolutions that have an alignment error less than a desired 
threshold. 

31. An apparatus comprising: 

a video format analyzer and synchronizer for identifying 
a first screen resolution for a display, the first screen 
resolution corresponding to a resolution setting for the 
display and for receiving a video signal to be displayed 
on the display having a video signal resolution; and 

a zoom control logic, coupled to the video foimat analyzer 
and synchronizer, for creating a second screen resolu- 
tion for displaying the received video signal on at least 
a portion of the display based on the video signal 
resolution, the first screen resolution and a comparison 
of the second screen resolution to the amount of 50 
memory available in a frame buffer for the display, to 
create the largest second screen resolution that can be 
used with the memory available. 

32. The apparatus of claim 31, wherein the apparatus 
comprises a video multiplexing system. 

33. The apparatus of claim 31, wherein the zoom control 
logic is also for altering a pixel width for each of a plurality 
of pixels of the received video signal. 

34. The apparatus of claim 33, wherein the zoom control 
logic is also for identifying a plurality of screen resolutions 
which can be used as the second screen resolution, and for 
selecting as the second screen resolution the one of the 
plurality of screen resolutions which provides a smallest 
alignment error with the first screen resolution. 

35. The apparatus of claim 33, wherein the zoom control 
logic is also for altering a pixel clock period corresponding 
to the second screen resolution. 



36. The apparatus of claim 33, wherein the zoom control 
logic is also for increasing the pixel clock period. 

37. The apparatus of claim 33, wherein the plurality of 
pixels comprises every pixel to be displayed as part of the 
image. 

38. The apparatus of claim 31, wherein the received video 
signal is displayed on the entire display. 

39. The apparatus of claim 31, further comprising a video 
switch, coupled to both the zoom control logic and the video 
format analyzer and synchronizer, for overlaying an image 
having the second screen resolution on an image having the 
first screen resolution. 

40. The apparatus of claim 31, wherein the received video 
signal includes one or more windows which provide video 
information from one or more video conferencing cameras. 

41. The apparatus of claim 31, wherein a horizontal 
resolution of the second screen resolution is less than a 
horizontal resolution of the first screen resolution. 

42. The apparatus of claim 31, wherein a horizontal 
resolution of the second screen resolution is greater than a 
horizontal resolution of the first screen resolution. 

43. The apparatus of claim 31, wherein the zoom control 
logic is also for generating a first candidate resolution and a 
second candidate resolution, comparing each candidate reso- 
lution to the available frame buffer memory, and selecting a 
candidate resolution based on the comparison. 

44. The apparatus of claim 43, wherein the zoom control 
logic is also for generating further candidate resolutions by 
incrementing the resolution by a selected amount. 

45. The apparatus of claim 43, wherein the zoom control 
logic is also for selecting the largest resolution that will fit 
into the available frame buffer memory. 

46. The apparatus of claim 31, wherein the zoom control 
logic is also for generating a first candidate resolution and a 
second candidate resolution, determining the alignment 
error between each candidate resolution and the first 
resolution, comparing each candidate resolution to the avail- 
able frame buffer memory, and selecting a candidate reso- 
lution with the largest resolution that will fit into the 
available frame buffer memory from among those candidate 
resolutions that have an alignment error less than a desired 
threshold. 

47. An apparatus comprising: 

means for identifying a first screen resolution for a 
display; 

means for receiving a video signal to be displayed on the 

display having a video signal resolution; and 
means, coupled to the means for identifying, a first screen 
resolution, for creating a second screen resolution for 
displaying the received video signal on the display 
based on the video signal resolution, the first screen 
resolution and a comparison of the second screen 
resolution to the amount of memory available in a 
frame buffer for the display, to create the largest second 
screen resolution that can be used with the memory 
available. 

48. The apparatus of claim 47, wherein the means for 
creating is also for altering a pixel width for each of a 
plurality of pixels of the received video signal. 

49. The apparatus of claim 48, wherein the means for 
60 creating is also for identifying a plurality of screen resolu- 
tions which can be used as the second screen resolution, and 
for selecting as the second screen resolution the one of the 
plurality of screen resolutions which provides a smallest 
alignment error with the first screen resolution. 

50. The apparatus of claim 48, wherein the means for 
creating is also for altering a pixel clock period correspond- 
ing to the second screen resolution. 
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51. The apparatus of claim 50, wherein the means for creating, for overlaying an image having the second screen 
creating is also for increasing the pixel clock period. resolution on an image having the first screen resolution. 

52. The apparatus of claim 47, further comprising means, 

coupled to both the means for identifying and the means for ***** 
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